This document is dedicated to conducting the confirmatory performance analyses that were proposed for Experiment 1.
Design. The design and analysis is a 2 (difficulty: harder than reference vs. easier than reference) X 2 (difference: moderate vs. extreme) X 2 (transition: repeat vs. switch) within-subjects ANOVA on RTs and error rates.
Predictions. There are predicted main effects of difficulty (easier deck has faster RTs and more errors than harder), and transition (typical switch cost). There is a predicted two-way interaction between difference and difficulty, such that harder decks are associated with slower RTs. There is another predicted two-way interaction between transition and difficulty, such that switch costs are expected to be greater for decks that are easier than reference. Finally, there is a predicted three-way interaction, where switch costs are expected to be larger for easier relative to harder decks. This is because switching tends to be more costly when it is done more infrequently.
Below is the cleaned data:
d <- read.csv('../../../data/dstClean.csv')
d <- d %>%
filter(transition != 'startBlock', chosenDeckId != 'reference')
N <- d %>%
group_by(subject) %>%
summarize(n()) %>%
nrow(.)
d
The sample size is 70.
cellMeans <- d %>%
group_by(subject, difference, difficulty, transition) %>%
summarize(mrt = mean(cuedRt)) %>%
group_by(difference, difficulty, transition) %>%
summarize(rt = mean(mrt), se = sd(mrt) / sqrt(n()))
cellMeans
cellMeans %>%
ggplot(aes(x = difference, y = rt, group = transition)) +
geom_bar(stat = 'identity', aes(fill = transition), color = 'black', position = position_dodge(width = 0.9)) +
geom_errorbar(aes(ymin = rt - se, ymax = rt + se), position = position_dodge(width = 0.9), width = 0.5) +
facet_wrap(~difficulty) +
theme_bw() +
xlab('Transition Type') +
ylab('Response Time (ms)') +
scale_fill_manual(name = 'Transition', values = c(`repeat` = 'Light Grey', switch = 'Black')) +
theme(strip.background = element_rect(color = 'black', fill = 'white'),
legend.position = 'top')
Subject-wise cell means
subjectCellMeans <- d %>%
group_by(subject, difference, difficulty, transition) %>%
summarize(rt = mean(cuedRt))
subjectCellMeans
Omnibus Model There are 15 subjects with missing cells in the design and they are being dropped from the analysis
badSubjects <- data.table(subjectCellMeans)[,.(count = .N), by = subject][count < 8, subject]
subjectCellMeans <- subjectCellMeans[!(subjectCellMeans$subject %in% badSubjects),]
m1 <- ezANOVA(wid = subject, within = .(difference, difficulty, transition), dv = rt, data = subjectCellMeans, detailed = TRUE)
## Warning: Converting "subject" to factor for ANOVA.
## Warning: You have removed one or more levels from variable "transition".
## Refactoring for ANOVA.
cs <- c(colnames(m1$ANOVA), 'n2p')
omni <- cbind(m1, data.frame(n2p = m1$ANOVA$SSn / (m1$ANOVA$SSn + m1$ANOVA$SSd)))
colnames(omni) <- cs
omni$p <- round(omni$p, 3)
omni
Marginal Means for Main Effects
d %>%
group_by(subject, difficulty) %>%
summarize(rtime = mean(cuedRt)) %>%
group_by(difficulty) %>%
summarize(rt = mean(rtime), se = sd(rtime) / sqrt(N))
d %>%
group_by(subject, transition) %>%
summarize(rtime = mean(cuedRt)) %>%
group_by(transition) %>%
summarize(rt = mean(rtime), se = sd(rtime) / sqrt(N))
Planned Comparisons
Difficulty X Difference
## difficulty X difference
DFd <- m1$ANOVA$DFd[5]
SSd <- m1$ANOVA$SSd[5]
MSe <- SSd / DFd
omnibusParams <- c('DFd' = DFd, 'SSd' = SSd, 'MSe' = MSe)
d %>%
group_by(subject, difference, difficulty) %>%
summarize(mrt = mean(cuedRt)) %>%
group_by(difference, difficulty) %>%
summarize(rt = mean(mrt), se = sd(mrt) / sqrt(n()))
m2 <- ezANOVA(wid = subject, within = difference, dv = rt, data = subjectCellMeans[subjectCellMeans$difficulty == 'Easier than Reference',], detailed = TRUE)
## Warning: Converting "subject" to factor for ANOVA.
## Warning: Collapsing data to cell means. *IF* the requested effects are a
## subset of the full design, you must use the "within_full" argument, else
## results may be inaccurate.
easier <- formatSimpleEffects(m2, omnibusParams)
m2 <- ezANOVA(wid = subject, within = difference, dv = rt, data = subjectCellMeans[subjectCellMeans$difficulty == 'Harder than reference',], detailed = TRUE)
## Warning: Converting "subject" to factor for ANOVA.
## Warning: Collapsing data to cell means. *IF* the requested effects are a
## subset of the full design, you must use the "within_full" argument, else
## results may be inaccurate.
harder <- formatSimpleEffects(m2, omnibusParams)
data.frame(Contrast = c('Simple effect of difference for Easier than Reference', 'Simple effect of difference for Harder than Reference'),
Result = c(easier, harder))
Transition X Difficulty
DFd <- m1$ANOVA$DFd[7]
SSd <- m1$ANOVA$SSd[7]
MSe <- SSd / DFd
omnibusParams <- c('DFd' = DFd, 'SSd' = SSd, 'MSe' = MSe)
subjectCellMeans %>%
group_by(subject, transition, difficulty) %>%
summarize(mrt = mean(rt)) %>%
group_by(transition, difficulty) %>%
summarize(rt = mean(mrt), se = sd(mrt) / sqrt(n()))
m2 <- ezANOVA(wid = subject, within = transition, dv = rt, data = subjectCellMeans[subjectCellMeans$difficulty == 'Easier than Reference',], detailed = TRUE)
## Warning: Converting "subject" to factor for ANOVA.
## Warning: You have removed one or more levels from variable "transition".
## Refactoring for ANOVA.
## Warning: Collapsing data to cell means. *IF* the requested effects are a
## subset of the full design, you must use the "within_full" argument, else
## results may be inaccurate.
easier <- formatSimpleEffects(m2, omnibusParams)
m2 <- ezANOVA(wid = subject, within = transition, dv = rt, data = subjectCellMeans[subjectCellMeans$difficulty == 'Harder than reference',], detailed = TRUE)
## Warning: Converting "subject" to factor for ANOVA.
## Warning: You have removed one or more levels from variable "transition".
## Refactoring for ANOVA.
## Warning: Collapsing data to cell means. *IF* the requested effects are a
## subset of the full design, you must use the "within_full" argument, else
## results may be inaccurate.
harder <- formatSimpleEffects(m2, omnibusParams)
data.frame(Contrast = c('Simple effect of transition for Easier than Reference', 'Simple effect of transition for Harder than Reference'),
Result = c(easier, harder))
Post-hoc follow ups on the three-way interaction
DFd <- m1$ANOVA$DFd[8]
SSd <- m1$ANOVA$SSd[8]
MSe <- SSd / DFd
omnibusParams <- c('DFd' = DFd, 'SSd' = SSd, 'MSe' = MSe)
m2 <- ezANOVA(wid = subject, within = .(transition, difference), dv = rt, data = subjectCellMeans[subjectCellMeans$difficulty == 'Easier than Reference',], detailed = TRUE)
## Warning: Converting "subject" to factor for ANOVA.
## Warning: You have removed one or more levels from variable "transition".
## Refactoring for ANOVA.
easier <- formatSimpleEffects(m2, omnibusParams)
m2 <- ezANOVA(wid = subject, within = .(transition, difference), dv = rt, data = subjectCellMeans[subjectCellMeans$difficulty == 'Harder than reference',], detailed = TRUE)
## Warning: Converting "subject" to factor for ANOVA.
## Warning: You have removed one or more levels from variable "transition".
## Refactoring for ANOVA.
harder <- formatSimpleEffects(m2, omnibusParams)
data.frame(Contrast = c('Simple effect of transition X difference for Easier than Reference', 'Simple effect of transition X difference for Harder than Reference'),
Result = c(easier, harder))
Testing for simple effect of difference across levels of transition (for easier than reference only)
SSd <- m1$ANOVA$SSd[2]
DFd <- m1$ANOVA$DFd[2]
MSe <- SSd / DFd
omnibusParams <- c('DFd' = DFd, 'SSd' = SSd, 'MSe' = MSe)
m2 <- ezANOVA(wid = subject, within = difference, dv = rt, data = subjectCellMeans[subjectCellMeans$difficulty == 'Easier than Reference' & subjectCellMeans$transition == 'repeat',], detailed = TRUE)
## Warning: Converting "subject" to factor for ANOVA.
repetition <- formatSimpleEffects(m2, omnibusParams)
m2 <- ezANOVA(wid = subject, within = difference, dv = rt, data = subjectCellMeans[subjectCellMeans$difficulty == 'Easier than Reference' & subjectCellMeans$transition == 'switch',], detailed = TRUE)
## Warning: Converting "subject" to factor for ANOVA.
Switch <- formatSimpleEffects(m2, omnibusParams)
data.frame(Contrast = c('Simple effect of difference for repetitions and Easier than Reference', 'Simple effect of difference for switches and Easier than Reference'),
Result = c(repetition, Switch))
Below is the cleaned data:
d <- read.csv('../../../data/dstCleanErrors.csv')
d <- d %>%
filter(transition != 'startBlock', chosenDeckId != 'reference')
d
cellMeans <- d %>%
filter(transition != 'startBlock') %>%
group_by(subject, difference, difficulty, transition) %>%
summarize(merror = mean(error)) %>%
group_by(difference, difficulty, transition) %>%
summarize(error = mean(merror), se = sd(merror) / sqrt(n()))
cellMeans
cellMeans %>%
ggplot(aes(x = difference, y = error, group = transition)) +
geom_bar(stat = 'identity', aes(fill = transition), color = 'black', position = position_dodge(width = 0.9)) +
geom_errorbar(aes(ymin = error - se, ymax = error + se), position = position_dodge(width = 0.9), width = 0.5) +
facet_wrap(~difficulty) +
theme_bw() +
xlab('Transition Type') +
ylab('Error Rate') +
scale_fill_manual(name = 'Transition', values = c(`repeat` = 'Light Grey', switch = 'Black')) +
theme(strip.background = element_rect(color = 'black', fill = 'white'),
legend.position = 'top')
Subject-wise cell means
subjectCellMeans <- d %>%
group_by(subject, difference, difficulty, transition) %>%
summarize(error = mean(error))
subjectCellMeans
Omnibus Model
badSubjects <- data.table(subjectCellMeans)[, .(count = .N), by = subject][count < 8, subject]
subjectCellMeans <- subjectCellMeans[!(subjectCellMeans$subject %in% badSubjects),]
m1 <- ezANOVA(wid = subject, within = .(difference, difficulty, transition), dv = error, data = subjectCellMeans, detailed = TRUE)
## Warning: Converting "subject" to factor for ANOVA.
## Warning: You have removed one or more levels from variable "transition".
## Refactoring for ANOVA.
cs <- c(colnames(m1$ANOVA), 'n2p')
omni <- cbind(m1, data.frame(n2p = m1$ANOVA$SSn / (m1$ANOVA$SSn + m1$ANOVA$SSd)))
colnames(omni) <- cs
omni
Marginal Means for Main Effects
print('Main effect of switch cost')
## [1] "Main effect of switch cost"
d %>%
group_by(transition, subject) %>%
summarize(err = mean(error)) %>%
group_by(transition) %>%
summarize(error = mean(err), se = sd(err) / sqrt(N))
print('Overall error rate')
## [1] "Overall error rate"
d %>%
group_by(subject) %>%
summarize(err = mean(error)) %>%
summarize(error = mean(err), se = sd(err) / sqrt(N))
Planned Comparisons
Difficulty X Difference
## difficulty X difference
DFd <- m1$ANOVA$DFd[5]
SSd <- m1$ANOVA$SSd[5]
MSe <- SSd / DFd
omnibusParams <- c('DFd' = DFd, 'SSd' = SSd, 'MSe' = MSe)
d %>%
group_by(subject, difference, difficulty) %>%
summarize(mrt = mean(cuedRt)) %>%
group_by(difference, difficulty) %>%
summarize(rt = mean(mrt), se = sd(mrt) / sqrt(n()))
m2 <- ezANOVA(wid = subject, within = difference, dv = error, data = subjectCellMeans[subjectCellMeans$difficulty == 'Easier than Reference',], detailed = TRUE)
## Warning: Converting "subject" to factor for ANOVA.
## Warning: Collapsing data to cell means. *IF* the requested effects are a
## subset of the full design, you must use the "within_full" argument, else
## results may be inaccurate.
easier <- formatSimpleEffects(m2, omnibusParams)
m2 <- ezANOVA(wid = subject, within = difference, dv = error, data = subjectCellMeans[subjectCellMeans$difficulty == 'Harder than reference',], detailed = TRUE)
## Warning: Converting "subject" to factor for ANOVA.
## Warning: Collapsing data to cell means. *IF* the requested effects are a
## subset of the full design, you must use the "within_full" argument, else
## results may be inaccurate.
harder <- formatSimpleEffects(m2, omnibusParams)
data.frame(Contrast = c('Simple effect of difference for Easier than Reference', 'Simple effect of difference for Harder than Reference'),
Result = c(easier, harder))
Transition X Difficulty
DFd <- m1$ANOVA$DFd[7]
SSd <- m1$ANOVA$SSd[7]
MSe <- SSd / DFd
d %>%
group_by(subject, transition, difficulty) %>%
summarize(mrt = mean(cuedRt)) %>%
group_by(transition, difficulty) %>%
summarize(rt = mean(mrt), se = sd(mrt) / sqrt(n()))
m2 <- ezANOVA(wid = subject, within = transition, dv = error, data = subjectCellMeans[subjectCellMeans$difficulty == 'Easier than Reference',], detailed = TRUE)
## Warning: Converting "subject" to factor for ANOVA.
## Warning: You have removed one or more levels from variable "transition".
## Refactoring for ANOVA.
## Warning: Collapsing data to cell means. *IF* the requested effects are a
## subset of the full design, you must use the "within_full" argument, else
## results may be inaccurate.
easier <- formatSimpleEffects(m2, omnibusParams)
m2 <- ezANOVA(wid = subject, within = transition, dv = error, data = subjectCellMeans[subjectCellMeans$difficulty == 'Harder than reference',], detailed = TRUE)
## Warning: Converting "subject" to factor for ANOVA.
## Warning: You have removed one or more levels from variable "transition".
## Refactoring for ANOVA.
## Warning: Collapsing data to cell means. *IF* the requested effects are a
## subset of the full design, you must use the "within_full" argument, else
## results may be inaccurate.
harder <- formatSimpleEffects(m2, omnibusParams)
data.frame(Contrast = c('Simple effect of transition for Easier than Reference', 'Simple effect of transition for Harder than Reference'),
Result = c(easier, harder))
I’ll deal with the three-way only if I need to.
A work by Dave Braun
dab414@lehigh.edu